Eecient Collective Communication on Heterogeneous Networks of Workstations 1

Networks of Workstations (NOW) have become an attractive alternative platform for high performance computing. Due to the commodity nature of workstations and interconnects, the NOW environments are being gradually redeened as Heterogeneous Networks of Workstations (HNOW) environments. This paper presents a new framework to implement collective communication operations (as deened by the Message Passing Interface (MPI) standard) eeciently for the emerging HNOW environments. We rst classify diierent types of heterogeneity in HNOW and then focus on one important characteristic: communication capabilities of workstations. By taking this characteristic into account, we propose a new Speed-Partitioned Ordered Chain (SPOC) approach to order the participating nodes of a given collective communication operation in an optimal fashion. Such an ordering allows exibility to implement collective communication operations with lower latency. Using the new approach, a set of tree-based algorithms are proposed for frequently used collective communication operations such as broadcast, multicast, and barrier synchronization. All these algorithms are compared with the standard MPI implementations on experimental as well as simulated testbeds. On a 16-node existing HNOW environment with SGI workstations and ATM interconnection, our approach reduces the latency of broadcast, multi-cast, and barrier synchronization operations by a factor of up to 1:4 compared to the existing MPICH implementation. On a 64-node simulated testbed, our approach promises to reduce the latency of broadcast operation by a factor of up to 4:8 compared to the MPI implementations. Improvements up to a factor of 3:3 and 2:3 are observed for multicast and barrier synchronization , respectively. Thus, these results demonstrate signiicant potential to be applied towards designing scalable collective communication libraries for current and future generation HNOW environments.